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Abstract 

l  } 

We  presents*  tutorial  describing  aspects  o£  the  coding  of^s inula Cions  of 
models  of  visual  cortical  development.  The  model  considered  has  an  anatomy  of  an 
excitatory  projection  from  thalamus  to  cortex  combined  with  intracortical 
inhibition.  Cortical  cells  develop  specificity  to  stimulus  patterns  in  this 
model  only  when  appropriate  experience  enables  synaptic  modification  to  organize 
tne  network. 

The  simulation  consists  of  a  time  loop.  For  each  iteration  of  this  loop,  a 
stimulus  is  generated,  the  cortical  response  to  this  stimulus  is  computed,  and 
synaptic  weights  are  modified.  The  developing  network  is  tested  intermittently 
and  tne  behavior  of  the  system  analyzed. 

Some  of  the  details  of  the  coding  given  include  a  method  of  describing 
rearing  conditions,  a  convenient  abstract  form  for  the  input  stimuli,  an 
iterative  calculation  of  the  intracortical  feedback,  a  simple  way  to  store 
synaptic  strengths,  and  routines  for  performing  the  analysis.  ■ 
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In  pares  1  and  II  of  this  series  (refs.  3.8),  we  discussed  the  applicability 
of  node ling  to  problems  in  visual  cortical  phyaiology,  and  described  some  of  the 
models  which  have  been  proposed,  concentrating  on  the  development  of  specificity 
in  neurons  receiving  excitatory  inputs  from  the  lateral  geniculate  nucleus  (LGN) 
and  inhibitory  inputs  from  other  cortical  units.  We  now  present  a  tutorial  in 
coding  simulations  of  such  a  model.  The  goal  of  these  simulations  is  to  provide 
tne  details  of  the  response  properties  of  the  elements  of  the  model  under  various 
conditions,  including  a  number  of  simulated  normal  and  abnormal  rearing 
conditions  and  over  a  range  of  parameter  values. 

OVERVIEW 

Experiment,  model,  and  simulation 

The  simulations  we  consider  mimic,  in  form,  classic  deprivation 
experiments.  Kittens  are  typically  reared  under  special  conditions,  and  a 
population  of  single  units  is  then  tested  physiologically  to  determine  their 
visual  response  properties.  The  goal  of  such  experiments  is  to  assess  the 
effects  of  different  rearing  conditions  on  the  response  properties  of  the 
cortical  units. 

The  second  paper  of  the  series  (ref.  8)  reviews  the  results  of  numerous 
experiments  on  the  neuronal  development  of  kitten  visual  cortex.  We  have 
concentrated  on  the  development  of  orientation  selectivity.  As  the  second  paper 
illustrates,  there  are  numerous  models  in  Che  literature  which  posit  mechanisms 
of  orientation  selectivity  and  attempt  to  show  how  visual  experience  might 
affect  components  of  these  mechanisms.  The  various  mechanisms  proposed  use 
different  schemes  of  excitation  and  inhibition  to  wire  up  a  network  of  cortical 
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cell*.  These  excitatory  and  inhibitory  connections  are  assumed  to  be  modified 
by  visual  experience  in  order  to  produce  the  alterations  of  response  properties 
seen  experimentally.  Although  there  ia  practically  no  direct  evidence  in 
mammalian  studies  for  the  hypothesis  that  it  ia  such  changes  in  wiring  which 
bring  about  the  changes  accessible  to  microelectrodes,  there  is  general 
acceptance  of  the  central  hypothesis  that  synapses  are  modified.  Most  models 
have  applied  versions  of  Hebb's  postulate  (5),  which  produce  connections  whose 
efficacy  is  related  to  the  correlation  between  pre-  and  post-synaptic 
activities. 

To  illustrate  some  techniques  in  coding  simulations  we  consider  a  model 
based  on  an  anatomy  of  excitatory  afferent  fibers  from  thalamus  to  cortex 
combined  with  inhibitory  intracortical  fibers.  The  afferent  fibers  are  divided 
into  two  groups:  fibers  relaying  signals  from  the  left  eye  and  fibers  relaying 
signals  from  the  right  eye.  Visual  stimuli  are  input  as  activities  on  the 
afferent  fibers  to  cortex.  These  stimuli  drive  the  cortical  cells,  determining, 
along  with  the  cortical  activities,  the  modification  of  the  synaptic 
connectivities. 

The  model  we  are  considering  can  be  summarized  by  the  following  equations. 


R  -  f(SA  -  QR) 

(1) 

dS/dt  -  g^(A,R,S;parameters) 

(2) 

dQ/dt  “  g^( R, R,Q; parameters) 

(3) 

A  (a  vector  of  dimension  equal  to  the  number  of  afferent  fibers,  M)  represents 
the  activity  in  the  fibers  afferent  to  cortex.  R  (a  vector  of  dimension  equal  to 
tne  number  of  cortical  cells,  N)  is  the  response  of  the  cortical  cells.  S  (an  N 
x  M  matrix)  and  Q  (an  N  x  N  matrix)  hold  the  synaptic  weights,  excitatory  from 
LON  and  inhibitory  from  cortex,  respectively,  onto  the  cortical  cell*.  We  let 


all  aynapcic  strengths  be  positive,  and  subtract  the  intracortical  inputs  in 
order  to  render  then  inhibitory,  f  is  a  function  describing  the  relationship 
between  tbe  inputs  to  cortical  cells  and  the  resultant  response:  this  function 
is  generally  chosen  to  be  sigmoidal,  incorporating  a  firing  threshold  and  a 
saturation  at  high  discharge  rates.  The  synaptic  modification  rules  are  given  by 
and  g^,  which  depend  on  tbe  pre-  and  post-synaptic  activities  (respectively  A 
and  R  for  the  atferent  and  R  and  R  for  the  intracortical  synapses) ,  possibly  on 
tne  current  values  of  the  weights,  and  additionally  on  a  number  of  parameters 
which  determine,  among  other  things,  how  rapidly  synaptic  modification 
progresses.  Although  various  models  choose  different  forms  for  the  function  f  in 
(1),  and  take  a  number  of  positions  on  what  the  inputs  A  consist  of,  the  chief 
distinction  between  these  models  is  the  choice  of  synaptic  modification  rules  g 
(see  Table  1  in  ref.  8).  One  can  choose  to  modify  only  the  excitatory  synapses, 
or  only  the  inhibitory  synapses,  or  both  -  in  either  the  same  way  or  in  different 
ways.  Fortunately,  this  significant  variation  in  models  of  the  form  (l)-(3)  can 
easily  be  handled  by  the  simulation  coding  we  will  describe. 

Computing  the  afferent  activity,  A 

Our  first  job  in  translating  the  model  into  code  is  to  compute  the  atterent 
input  A.  Mow  the  visual  inputs  to  cortex  are  determined  by  the  rearing 
conditions.  Thus,  different  rearing  conditions  lead  to  different  cortical 
activity  patterns,  affecting  synaptic  modification,  resulting  in  tbe 
development  of  different  connectivities  and  hence  to  different  cortical  response 
properties.  Our  code  tbertore  starts  by  allowing  the  desired  rearing  conditions 
to  be  specified.  Since  this  is  a  major  independent  variable  in  running  simulated 
experiments,  we  code  the  resring  conditions  in  an  accessible  ("English  language") 


form.  For  instance,  in  order  to  perform  an  experiment  on  the  results  of 
monocular  deprivation,  one  simply  instructs  the  progrem  to  close  one  eye. 

The  next  step  in  the  code  is  to  take  the  given  rearing  condition,  and  to 
translate  tnis  simple  English  language  description  into  appropriate  simulated 
visual  stimuli  (the  variable  A).  Such  stimuli  can  be  thought  of  as  activities 
on  a  set  of  afferent  axons,  and  are  coded  as  a  vector,  with  the  components  of  the 
vector  corresponding  to  the  activities  on  individual  axons.  Thus,  monocular 
deprivation  by  closing  the  left  eye  will  lead  to  a  vector  with  some  pattern  of 
activities  in  the  components  corresponding  to  the  right  eye.  but  with  only  noise 
in  tbe  portion  of  the  vector  corresponding  to  the  left  eye.  At  each  moment  of 
simulated  time,  a  new  stimulus  is  generated  using  the  general  description  of  the 
rearing  conditions,  implying  that  the  simulated  visual  stimuli  continually  change 
as  in  real  life. 

Before  continuing  to  deacribe  the  sequence  of  routines  in  the  program,  we 
pause  to  discuss  the  crucial  nature  of  the  input  coding.  Despite  our  appeal 
above  to  "real  life",  the  key  to  successful  coding  of  a  simulated  visual 
environment  is  an  appreciation  of  the  necessity  of  abstraction.  The  challenge  in 
implementing  a  simulation  of  a  neural  model  is  to  create  a  formal  structure  which 
captures  the  essence  of  the  real  structures  being  modeled.  The  visual  world  is 
astoundingly  complex:  digitized  video  images  of  it  are  merely  a  shadow  of  tbe 
reality.  One  might  be  tempted  to  provide  "real"  input  to  a  simulated  cortex  by 
hooking  up  a  camera  to  a  computer,  but  this  would  neither  mimic  any  neural  events 
nor  allow  understanding  and  control  of  the  input  end  of  the  model.  We  will  not 
try  to  create  a  particularly  rich  imitation  of  tbe  visual  world,  but  instead  will 
try  to  faithfully  reproduce  the  few  aspects  of  tbe  retinal  images  with  which  we 
choose  to  be  concerned,  such  as  orientation. 

Host  of  tbe  models  described  in  ref.  8  arose  from  the  excitement  generated 


by  a  number  of  experiments  in  which  visual  experience  was  restricted  to  a  very 
limited  range  of  oriented  contours  (see  citations  in  ref.  8).  Kittens  were 
raised,  for  example,  viewing  only  horizontal  or  only  vertical  lines.  Modelers 
tnus  abstracted  out  oriented  lines  as  an  important  feature  of  the  visual 
environment,  whose  presentation  could  be  controlled.  We  suppose  then  that  tne 
environment  in  our  model  consists  of  brief  presentations  of  variously  oriented 
bars.  Such  a  bar  in  visual  space  can  reasonably  be  expected  to  stimulate  a  set 
of  retinal  ganglion  cells  which  form  a  bar  on  the  retinal  surface.  Assuming  a 
simple  retmotopic  projection  to  cortex  on  the  scale  of  this  bar  (not 
necessarily  a  valid  presumption,  since  it  essentially  amounts  to  considering 
everything  between  retina  and  cortex  as  strictly  relay  mechanisms),  we  can 
imagine  that  a  sheet  of  receptors  encodes  the  visual  stimuli  by  determining  which 
receptors  are  covered  by  a  bar  (Figure  1).  Different  orientations  all  stimulate 
a  central  cell,  corresponding  to  the  axis  of  rotation  of  the  family  of  bars,  but 
differ  m  the  periphery  of  the  sheet. 

The  activities  of  the  M  receptor  elements  in  the  sheet  can  be  thought  of  as 
forming  the  components  of  a  vector  of  dimension  M.  The  environment  of  various 
oriented  bars  can  be  implemented  as  a  set  of  such  vectors.  A  vertically  oriented 
bar  stimulates  certain  receptors  and  is  coded  into  a  corresponding  vector,  while 
a  horizontal  bar  corresponds  to  a  different  vector.  As  the  orientation  of  the 
visual  stimulus  rotates  from  horizontal  to  vertical  and  back  to  horizontal,  the 
vectors  corresponding  to  these  orientations  rotate  through  the  abstract  space  in 
which  they  live.  Thinking  of  these  vectors  as  directed  arrows,  the  arrowhead 
would  trace  out  a  circle  as  the  orientation  varies  through  180  degrees.  (We  say 
that  the  set  of  vectors  corresponding  to  oriented  stimuli  has  a  circular 
structure:  the  vectors  in  it  are  permutations  of  each  other  and  all  lie  on  a 

circle  in  their  possibly  high-dimensional  space,  since  the  order  in  which  the 


receptors  ere  listed  in  the  vector  is  arbitrary,  and  the  stuauli  are  all 
identical  except  in  terns  of  which  of  the  receptors  are  stianilated.)  Thus, 
although  it  is  certainly  feasible  and  sometimes  helpful  for  graphical  purposes  to 
represent  the  environment  in  terms  of  a  two-dimensional  sheet  of  a  large  number 
of  cells,  as  long  as  we  are  considering  only  orientation  we  do  not  need  the 
additional  possibilities  afforded  by  two  dimensions  (such  as  being  able  to  encode 
position,  length,  width,  or  movement).  The  orientation  domain  is  one-dimens lonal 
and  periodic  and  can  be  represented  by  any  circular  set  of  vectors.  Trivial 
examples  of  some  circular  sets  of  vectors  in  three  dimensions  are  {(1,0,0), 

(0,1 ,0) ,(0,0 ,1)>  or  {(1 ,1 ,0)  ,(0,1 ,1) ,(1 ,0 ,1)).  We  will  want  to  derive  more 
userul  sets  in  higher  dimensions,  without  necessarily  resorting  to  the  scheme 
mentioned  above  where  bars  were  projected  onto  a  sheet  of  cells. 

It  turns  out  that  the  important  parameter  in  defining  the  simulated 
environment  determines  how  close  together  the  input  patterns  lie  on  the  circle. 
(Translating  back  to  the  real  world,  what  matters  is,  how  similar  do  various 
oriented  bars  look  at  the  level  of  the  input  to  cortex?)  Our  approach  to 
constructing  input  patterns  on  the  computer  might  be  clarified  by  way  of  an 
analogy  to  color  mixing.  If  we  want  to  construct  the  color  yellow,  we  can  do  so 
by  mixing  red  and  green  light.  What  we  actually  mean  is  that  we  can  create  the 
perceptual  color  yellow  by  mixing  the  spectral  colors  red  and  green.  A  spectral 
color  is  simply  monochromatic  light,  and  is  best  described  by  the  wavelength. 
However,  we  make  an  assignment  from  wavelength  to  color  name,  for  convenience. 
Mow,  4iffetant  perceptual  colors  can  have  very  similar  or  very  different  spectral 
compositions:  yellow  and  orange  have  similar  compositions  in  the  form  of  a  banana 
and  an  orange  illuminated  by  white  light,  while  they  have  quite  different 


compositions  in  the  form  of  a  white  piece  of  paper  illuminated  with  monochromatic 
light  at  580  nm  (yellow)  and  600  nm  (orange)  wavelengths.  We  want  to  control 


Cue  aiaii.ari.cy  of  our  input  paCCerna  in  just  thia  manner.  We  atarC  with  aoae 
abatracc  tokena  (unit  vectora)  which  are  aaaigned  orientation  naaea  analogoua  to 
tne  way  that  wavelengtha  are  aaaigned  color  naaea.  We  then  mix  theae  tokena  in 
varioua  aaounta  to  create  more  of  one  orientation  than  of  othera  in  our  final 
input  pattern,  which  thereby  "looka  like"  the  orientation  of  the  dominant 
token.  If  we  were  to  uae  only  a  aingle  token  at  a  time,  different  input  patterns 
would  conaist  of  completely  different  tokena.  By  mixing  together  tokena  which 
represent  similar  orientations,  we  produce  "oriented"  patterns  which  overlap  in 
their  distribution  of  tokens,  despite  having  different  nuabers  of  each  type  of 
token.  We  retain  color  naaea,  or  orientation  naaea,  for  the  tokens  aa  well  as 
for  the  mixtures  because  the  correspondence  is  convenient,  once  we  understand  the 
possible  confusion. 

We  have  replaced  the  two-dimensional  sheet  of  receptors  by  a  circle  of  input 
elements.  These  elements  will  be  our  tokens,  analogous  to  spectral  colors.  Each 
.ement  on  the  circle  carries  an  activity,  and  the  afferent  input  to  cortex 
consists  of  the  vector  formed  by  the  activities  in  these  components.  We  generate 
a  stimulus  vector  by  first  designating  one  of  the  input  elements  on  the  circle 
as  representing  the  desired  orientation.  For  definiteness,  say  the  input 
elements  are  indexed  by  1  through  10,  with  1  representing  vertical  and  6 
representing  horizontal.  Say  we  want  to  create  a  vector  coding  a  vertical  bar. 

We  designate  element  number  1  as  our  "center"  component.  The  stimulus  vector  is 
given  a  large  amplitude  in  this  component,  indicating  that  the  vector 
corresponds  to  this  orientation.  Neighboring  components  on  the  circle  (that  is, 
elements  indexed  by  2,  3,  9,  and  10)  are  given  smaller  amplitudes,  indicating 
that  those  orientations  are  similar  but  not  identical.  As  we  move  around  the 
circle  of  input  elements,  we  assign  amplitudes  which  reflect  how  similar  the 
orientation  represented  by  each  element  is  to  the  desired  center  orientation. 


Components  which  represent  similar  orientations  are  active,  while  elements 
representing  orientations  orthogonal  to  the  center  orientation  (elements  4,  5,  6, 
7,  and  8)  are  relatively  inactive. 

Computationally,  we  use  a  function  which  is  maximal  at  the  center  element 
and  decreases  away  from  this  center  (Pigure  2).  This  function  sets  the 
activities  in  each  of  the  input  elements*  The  width  of  this  function  controls 
the  overlap  between  patterns  in  a  straightforward  way.  Returning  to  the  color 
analogy,  this  function  is  analogous  to  the  spectrum  of  the  illumination,  with  the 
width  corresponding  to  the  bandwidth.  If  the  function  is  parametrized  as  very 
narrow,  then  different  patterns  would  overlap  very  little.  (Interpreting  back  to 
physiology,  the  patterns  of  firing  in  the  optic  radiations  afferent  to  a  cortical 
cell  would  be  quite  different  for  a  vertical  bar  and  a  bar  inclined  10  degrees 
toward  horizontal.)  If  the  function  decreases  slowly,  all  patterns  would  be  quite 
similar.  Two  or  three  lines  of  code  ( 'F0NCTI0N  contour'  in  the  code  given  below) 
suffice  to  calculate  the  input  vector  with  such  a  system,  and  it  carries  with  it 
the  graphical  form  of  a  tuning  curve  which  can  be  compared  with  the  output  tuning 
curve,  since  each  input  element  corresponds  to  the  pattern  centered  on  it.  We 
have  moved  somewhat  far  from  the  reality  of  oriented  contours,  however,  and  we 
must  keep  in  mind  that  our  construct  is  an  abstract  model,  and  not  a  realistic 
model,  of  the  inputs  to  visual  cortex.  That  is,  the  components  of  our  stimulim 
sector  iSL  aat  corxeapoad  directly  actual  fibers  afferent  *  cortical  cell 
(but  we  will  still  refer  to  them  as  "afferent  fibers"  because  of  the  anatomical 
analogy) . 

Computing  the  corticaL  response,  R 

Once  the  activity  on  the  afferent  fibers  is  determined,  we  can  compute  the 
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activity  in  each  cortical  cell  by  integrating  the  atferent  inputs  to  each  cell 
with  the  intracortical  inputs  (equation  1).  To  code  this  aspect  of  the  model  ve 
write  the  synaptic  weights  to  a  cortical  cell  as  the  components  of  a  synaptic 
weight  vector.  To  determine  the  postsynaptic  potentials  (SA  -  QR)  of  the 
cortical  cells  we  take  the  product  of  the  synaptic  weights  (S  and  -Q)  with  a 
vector  which  is  formed  from  the  components  of  the  input  vector  to  the  afferent 
fibers  and  the  otner  cortical  cell  activities  coming  in  along  the  intracortical 
fibers  (A  and  R).  f(SA  -  QR)  determines  the  actual  cell  activity  R,  where  the 
function  f  comes  from  some  model  for  the  firing  behavior  of  kitten  cortical 
neurons  (usually  f  incorporates  a  firing  threshold  and  firing  saturation,  with  a 
somewhat  linear  range  in  between).  Because  the  intracortical  inputs  depend  on 
tne  activity  in  other  cortical  cells  which  are  being  computed  simultaneously 
(that  is,  our  computation  of  R  requires  that  we  know  R  already),  we  have  a 
problem.  Our  system  is  a  recurrent,  feedback  type  system,  and  because  of  size, 
nonlinearities,  and  a  desire  to  mimic  reality,  we  do  not  compute  the  activity 
explicitly,  but  instead  perform  an  iterative  calculation  which  approximates  the 
steady  state  activity.  Through  such  computations  (described  below)  we  derive  an 
activity  for  each  cortical  cell.  Writing  all  the  cortical  cell  activities  as  a 
vector,  we  have  the  output  of  the  simulation  in  raw  form. 

Computing  the  synaptic  modifications,  dS/dt  and  dQ/dt 

We  can  then  use  the  cortical  activity  to  drive  synaptic  modifications,  which 
are  the  result  of  the  particular  visual  stimulus  which  was  experienced  at  the 
most  recent  moment  of  time.  Coding  the  synaptic  modifications  amounts  to  writing 
the  model's  modification  rule  in  functional  form,  which  usually  takes  no  more 
than  a  few  lines  of  code. 


Simulating  viaual  experience 


The  code  for  generating  a  stimulus,  computing  the  cortical  response,  and 
modifying  the  synapses  falls  within  a  simulated  time  loop  which  is  the  sun 
driver  of  the  simulation.  On  each  pass  of  the  time  loop  a  potentially  different 
stimulus  from  the  environment  which  is  consistent  with  the  specified  rearing 
conditions,  is  presented  through  the  set  of  simulated  afferent  fibers.  (We  use  a 
random  variable  to  pick  the  order  in  which  the  stimuli  are  presented  to  the 
system.)  Passing  through  the  simulated  time  loop  thus  models  the  experimentally 
desirable  aspect  of  a  kitten's  rearing  environment,  which  consista  of  a 
controlled  set  of  visual  atimuli  that  the  kitten  ia  exposed  to  in  real  time  in 
some  order,  possibly  random.  In  fact,  experimental  control  over  a  kitten's 
visual  experience  is  distressingly  poor.  A  simulation  can  take  into  account  some 
of  the  leas  controlled  aspects  of  this  experience,  by,  for  instance,  modeling 
moments  of  sleep  or  inattention  and  the  like  by  noisy,  rather  than  patterned, 
visual  inputs.  Rather  than  presenting  a  rigid  set  of  oriented  patterns,  with  the 
inputs  to  the  two  eyes  identical,  we  prefer  to  present  some  statistical 
distribution  of  a  large  set  of  stimuli  which  includes  occasional  unpatterned 
inputs  and  differing  patterns  in  the  two  eyes.  As  simulated  time  proceeds,  the 
various  stimuli  rewire  the  network,  with  a  speed  which  depends  on  the  speed  of 
synaptic  modifications  and  on  the  the  rate  at  which  stimuli  which  strongly  atfect 
synapses  appear  in  the  environment. 

Analysing  the  cortical  response  properties 

The  above  process  of  generating  a  stimulus,  computing  the  cortical  response, 
and  modifying  synapses,  is  occasionally  interrupted  in  order  to  test  the  network 


CODING 


The  following  example  of  «  progrea  deeigoed  to  simulate  e  visual  cortical 
development  model  will  illustrate  the  detaiia  involved  in  coding  such  a 
simulation*  We  have  written  versions  of  this  program  in  BASIL,  FORTRAN,  and 
PASCAL  for  various  machines,  including  LM2 ,  AMDAHL,  IBM,  VAX,  and  APOLLO,  and 
have  found  that  modifications  are  easily  made,  and  usually  have  proven  useful  in 
enriching  the  code*  We  present  a  PASCAL  version  (see  ref*  4  for  a  good 
description  of  this  language) ,  some  of  which  we  give  in  pseudocode  to  sketch  tne 
structure,  leaving  out  some  details  and  all  of  the  machine-dependent  features* 
The  omissions  include  the  graphics  routines,  which  we  feel  are  crucial  in 
understanding  the  performance  of  the  simulations,  but  which  must  be  created 
according  to  one's  own  resources  and  purposes.  Complete  source  codes  in  PASCAL 
(for  ArOLLO)  are  available  from  the  authors,  although  these  programs  are  specific 
examples  of  the  general  codings  described  in  this  article. 

We  simulate  processes  occurring  through  time,  so  the  program  consists  of  a 
time  loop.  Each  moment  of  time  corresponds  to  a  stimulus  presentation.  One 
could  conceivably  relate  this  simulated  time  unit  to  some  real  time  unit,  by 
reasoning  that  real  visual  stimuli  capable  of  driving  synaptic  modifications 
occur  with  a  certain  duration  and  rate  (ref.  1).  The  rate  of  change  of  synaptic 
strength  with  individual  stimulus  presentations  would  be  the  appropriate  quantity 
to  measure  in  the  real  world,  and  could  then  be  set  in  the  simulation  as  a 
parameter  in  the  synaptic  modification  routines  (in  the  example  of  a  synaptic 
modification  rule  given  below  this  is  the  time  constant  'h').  The  stead v-atate 
response  to  each  stimulus  presentation  is  used  to  perform  synaptic 
modification.  More  complex  simulations  might  reflect  a  less  discrete  view  of 
time,  allowing  for  a  smoothly  varying  stimulus  and  a  dynamic  response,  simply  by 
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with  a  standard  sec  of  stimuli.  These  teeC  sessions  allow  Che  evolution  of  the 
network  to  be  evaluated,  so  that  the  responses  of  cortical  cells  can  be  compared 
at  different  points  in  their  experience.  Bach  teat  seasion  consists  of  three 
main  parts:  generating  a  stimulus,  computing  responses,  and  analysing  the 
responses.  The  analysis  takes  the  raw  data  and  compiles  it  into  more  interesting 
measures  of  performance,  such  as  indices  of  selectivity  and  ocular  dominance  and 
population  statistics.  Note  that  no  synaptic  modification  is  performed  during 
test  sessions.  How  often  one  tests  the  system  depends  on  the  detail  desired  and 
on  tne  speed  at  which  the  network  is  changing. 


Summary  of  overview 


To  summarize,  the  structure  of  the  simulation  comprises  a  loop  indexed  by 
simulated  time,  during  each  iteration  of  which  several  procedures  are  cilled. 
First,  a  stimulus  is  chosen  according  to  the  desired  rearing  conditions.  Next, 
tbis  stimulus  is  translated  into  an  input  vector.  The  cortical  response  is  then 
computed.  Finally,  synaptic  strengths  are  modified.  A  branch  to  a  test  session 
is  made  at  desired  intervals  (Figure  3). 

Before  proceeding  to  a  more  detailed  examination  of  explicit  coding 
techniques,  we  would  like  to  emphasize  that  coding  the  visual  environment  of  the 
kitten  is  the  most  demanding  task  in  constructing  the  simulation.  A  model's 
anatomy  for  the  kitten  cortex  and  rule  for  synaptic  modification  imply  the  logic 
of  tn^  code  tbat  one  must  write.  Capturing  the  essential  characteristics  of  the 
kitten's  actual  environment  in  code,  however,  is  not  straightforward.  The  amount 
of  discussion  in  the  overview,  as  well  as  the  large  number  of  lines  in  the  code, 
devoted  to  tbis  one  task  attests  to  the  importance  of  this  task  in  simulating  the 
present  model  for  tbe  neuronal  development  of  kitten  cortex. 


inserting  an  inner  time  loop. 

Note  that  for  purpoaea  of  clarity,  ve  paaa  parameters  between  routinea 
explicitly  even  when  it  is  neither  necessary  nor  desirable.  (Explicitly  passed 
parameters  are  listed  in  the  parentheses  to  the  right  of  the  called  routine,  both 
in  the  calling  statement  and  in  the  definition  for  the  routine.  If  the  called 
routine  changes  the  value  of  a  parameter,  then  that  parameter  is  defined  as  a 
variable  in  the  parentheses  associated  with  the  routine  definition.)  In  general 


the  code  given  below  is 


to  make  it  understandable  to  the  reader. 


1-time .  but  instead  we  have  attempted 


MAII  PROGRAM  LOOP 

Figure  4  gives  the  driver  for  the  program,  which  generates  an  input, 
calculates  the  output,  modifies  synapses,  and  analyzes  the  responses.  Prior  to 
entering  this  principal  portion  of  the  program,  the  code  must  initialize 
variables  and  interact  with  the  user,  but  we  omit  those  important  but  tedious 
preliminaries  for  the  sake  of  brevity. 

STIMULUS  CKHKRAXIOR 


The  ij 


in  many  simulation  experiments  is  the 


environment.  In  order  to  easily  handle  the  various  rearing  conditions  in  the 
most  convenient  way,  we  have  coded  the  stimuli  in  plain  English.  The  routine  in 
Figure  S  determines  the  current  stimulus  as  a  function  of  time  and  a  random 
variable.  For  this  example,  we  simulate  an  experiment  where  the  early  rearing 
(prior  to  time  1000)  is  in  the  dark,  followed  by  rearing  in  the  light  assuming 
that:  20Z  of  experience  is  without  visual  input  (stimuli  consisting  of  noise). 


20X  consist*  of  patterns  through  only  one  eye  (either  the  right  or  the  left). 

10X  has  uncorrelated  patterns  in  the  two  eyes  (as  night  be  found  in  a  strabismic 
animal),  and  the  remaining  50Z  presents  similar  (although  not  always  identical) 


patterns  to  the  two  eyes.  Other  rearing  conditions  that  might  be  used  are 
'normalrearing',  'reverse  suture',  'adapting',  and  'sinusoid',  which  we^wiil 
describe  shortly.  Different  simulation  experiments  are  run  by  inserting  the 
desired  rearing  conditions  into  the  above  procedure.  This  process  can  be  made 
"user-friendly"  by  coding  the  procedure  as  a  menu-driven  interactive  choice 
paradigm. 

The  integer  'randon^_seed'  is  uniformly  distributed  over  a  large  interval,  so 
that  we  can  obtain  random  variables  uniformly  distributed  over  arbitrary 
intervals  as  above  by  applying  the  'MOD'  function,  which  returns  the  integer 
remainder  upon  division  by  the  modulus  (10  in  the  example  here). 

The  individual  stimulus-setting  routines  that  can  occur  in  the  'PROCEDURE 
Choose_stimulus'  follow  (Figures  7-9).  They  fill  in  the  components  of  the 
variable  'stimulus' ,  which  has  the  type  definition  given  in  Figure  6.  This 
particular  PASCAL-dependent  construction  provides  a  convenient  way  to  tranalate 
tne  "English-language"  rearing  conditions  into  code,  but  could  be  replaced  with  a 
series  of  conditional  statements  or  arbitrary  assignments  in  other  programming 
languagea.  The  type: 

rearing  “  ( correlated, ad, rs,dr, disparity, strabismus, adaptation, per iodic) 

consista  of  these  tags  which  refer  to  the  various  rearing  conditions: 

'correlated'  is  used  to  provide  identically  corresponding  patterns  in  the  two 
eyes,  'md'  means  monocular  deprivation  and  presents  a  pattern  in  the  open  eye  but 
only  noise  in  the  closed  eye,  'rs'  stands  for  reverse  suture  which  comprises 
successive  monocular  deprivations,  'dr'  abbreviates  dark  rearing  which  simply 
presents  noisy  inpucs  to  both  eyes,  'disparity'  gives  partly  correlated  patterns 
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to  the  two  eyes,  'strabismus'  gives  uncorrelated  patterns  to  the  two  eyes, 
'adaptation'  sets  a  particular  stimulus  for  repeated  presentation  to  either  eye 
alone  or  both  eyes  together,  and  'periodic'  allows  for  presentation  of  periodic 
patterns  ("gratings"),  rather  than  "bars”.  The  other  type  definitions  are: 
angle  ■  1..  numangles,  where  numangles  is  the  number  of  patterns  in  the  simulated 
visual  environment,  and  eyes  “  ( left, right, both) . 

The  routines  in  Figures  7-9  produce  the  desired  sistulated  experience  in  an 
understandable,  "English-language"  form,  by  taking  advantage  of  the 
record-variable  'stimulus'.  We  must  now  translate  this  information  into  a 
computational,  numerical  form,  namely  an  array  of  values  for  the  activities  in 
tne  axferent  fibers.  Active  fibers  will  be  assigned  numbers  near  1,  and  inactive 
fibers  numbers  near  0.  Thus,  it  the  visual  exposure  consists  cf  monocular 
deprivation,  with  the  left  eye  closed  and  the  right  eye  viewing  a  pattern,  then 
fibers  from  the  left  eye  will  carry  small  random  values  white  the  right  eye 
atferents  will  be  given  values  determined  by  the  pattern.  The  procedure  in 
Figure  lu  and  the  functions  it  calls  (Figures  11  and  12)  perform  this 
translation. 


USFOISK  CALC0LATI0R 


At  this  point  we  have  set  up  the  input  and  we  are  ready  to  turn  our 
attention  at  last  to  the  model  we  want  to  simulate.  The  input  has  been  coded 
into  the  vector  'af ferent.activity'  (A  in  equation  1)  and  will  now  be  fed  to  the 
cortical  cells,  which  will  respond  according  to  the  state  of  their  synapses. 

This  response  will  then  be  used  to  alter  the  synaptic  state. 

We  must  solve  equation  1,  B»f(SA-QR).  The  variables  S,  A,  and  Q  are  fixed 


here,  with  our  only  problem  being  that  we  must  deal  with  the  intracortical 


feedback  which  makes  B  a  function  of  itself.  While  negative  feedback  provides 
stability  to  the  visual  system  here,  from  a  strictly  computational  point  of  view 
this  feedback  can  cause  instabilities  when  pertorming  the  iterative  calculations 
needed  to  solve  the  equation.  For  instance,  it  one  starts  with  R  being  very 
small,  tnere  will  be  very  little  intracortical  inhibition  (QR  will  be  small). 

But  tnen  one  could  reason  that  the  lack  of  inhibition  would  lead  to 
overexcitation,  which  in  turn  would  lead  to  strong  inhibition  which  would  result 
in  little  response,  and  so  on  through  a  possibly  growing  oscillation.  This 
simple-minded  iterative  scheme,  which  can  be  written  in  scalar  form  as 

*  -  hv.» 

is  unstable  for  h>l .  The  constant  h  corresponds  to  the  eigenvalues  of  the  matrix 
Q  in  equation  1,  and  the  difficulty  of  ensuring  that  the  eigenvalues  of  Q  remain 
less  than  1  prevents  the  application  of  this  staple  explicit  one-step  scheme 
(where  new  response  values  are  computed  based  solely  on  the  values  from  the  last 
iteration) . 

Multi-step  schemes,  such  as 
y„-  a  -  h(yrt_a*  *,_,)/ 2. 

can  help  but  still  fail  to  prevent  oscillations  for  large  h.  What  is  needed 
instead  is  an  implicit  scheme  (where  the  current  iteration  appears  on  both  sides 
of  the  equation) ,  for  example  of  the  form 
yn-  a  -  h(y^.,+  y*)/2. 

This  method  is  stable  for  all  values  of  h.  Solving  such  implicit  schemes  exactly 
can  be  done  for  relatively  small  linear  systems,  but  is  out  of  the  question  for 
our  large,  often  nonlinear  equations.  Instead,  a  surprisingly  simple  method  can 
often  give  satisfactory  results:  update  individual  cells  from  iteration  n-1  to 
iteration  n,  while  other  cells  remain  in  iteration  n-1.  That  is,  the  updating  is 
asynchronous,  not  all  cells  are  upoated  at  once.  One  advantage  of  this  method  is 
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tnat  only  one  vector  of  activities  need  be  stored,  rather  than  two  (the  n-l'th 
and  n'th)  as  in  the  scheme  above.  At  any  point  in  the  computation,  some  of  the 
components  of  this  vector  will  belong  to  the  current  iteration  (the  a'tb)  while 
otnera  will  belong  to  the  last  iteration  (the  n-l'th).  The  advantage  in 
conserving  memory  ia  usually  not  great  in  these  days  of  large  virtual  memory.  He 
should  note  here  that  these  iterative  methods  also  tend  to  be  slow:  it  feedback 
can  be  avoided  in  a  model,  the  simulations  will  run  much  faster.  Civen  the 
necessity  of  simulating  a  recurrent  network,  however,  this  method  seems  to  be  as 
stable,  convenient,  and  fast  as  possible. 

The  usefulness  of  this  method  comes  from  the  fact  that  fluctuations  produced 
by  tne  iteration  process  are  averaged  out  by  adding  the  two  consecutive 
responses.  Clearly,  for  this  process  to  work  some  averaging  across  different 
cells  is  needed.  For  instance,  starting  from  very  low  activities  again,  one 
might  expect  the  first  few  cells  which  are  updated  to  respond  strongly  because  of 
the  lack  of  inhibition.  These  cells  would  then  provide  strong  inhibition  to 
later  cells,  which  would  never  respond.  The  result  would  be  an  unintended 
separation  of  the  cortex  into  those  cells  which  were  updated  early,  and  are 
active,  and  those  cells  updated  later,  which  are  inactive.  One  way  to  avoid  this 
problem  is  to  update  cells  in  random  order,  so  that  although  during  the  first 
iteration  the  early  cells  might  have  an  advantage  (or  disadvantage,  depending  on 
tne  initial  state),  there  ia  no  longer  any  meaning  to  early  and  late  atter  the 
first  round  of  updates.  A  cell  which  saw  little  inhibition  when  it  was  first 
updated  could  be  updated  again  soon  thereat  ter,  when  the  early  cells  are 
providing  strong  inhibition.  A  large  enough  sample  of  cells  is  needed  to  allow 
tnis  random  ordering  to  effectively  compute  averages  over  each  type  of  cortical 
response  property. 

The  procedure  in  Figure  13  provides  a  coding  of  this  random,  asynchronous. 
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iterative  method.  Since  the  iterative  proceaa  changea  only  the  intracortical 
input ,  ve  only  need  to  compute  the  afferent  input  to  each  cell  once.  The 
conoition  'atferent_input_hasnt_already_been_computed(cell) '  ia  a imply  given  by 
tne  boolean  array  'used':  af ferent_input_hasnt_already_been_computed(celi)  :■ 
NUT  uaed[cell].  Each  time  we  choose  a  cell  ve  set  'used'  to  TROE  for  that  cell, 
and  ve  choose  to  iterate  until  all  the  cells  have  been  chosen  at  least  once. 

This  means  that  at  least  one  cell  will  only  pass  through  this  iterative  procedure 
once,  buc  unless  a  cell  is  chosen  only  in  the  early  stages  (which  has  a  low 
probability)  there  is  no  harm  done.  It  is  important  to  keep  in  mind  that  only 
the  feedback  (QR  in  equation  1)  ia  being  forced  to  converge  here,  and  that  this 
is  a  global  property:  ve  are  not  really  forcing  each  cortical  cell  to  converge 
separately  and  independently  of  other  cells  (if  one  cell  is  less  active  and 
another  more  active  than  the  ideal  levels,  the  method  will  still  work).  Only 
the  total  cortical  activity,  which  behaves  like  the  average  activity,  must 
stabilize.  This  globality  lets  us  take  advantage  of  the  averaging  over  many 
corcical  cells.  Simulations  which  generate  specificity  also  provide  a  strong 
organizing  influence  which  tends  to  overcome  the  relatively  small  biasing  Errors 
from  the  iterative  computations,  as  cells  learn  to  respond  to  the  input  pattern 
without  regard  to  where  they  lie  in  the  updating  sequence. 

The  routine  in  Figure  13  calls  separate  procedures  for  afferent  and 
intracortical  inputs,  but  they  are  essentially  identical  calls  to  an  inner 
product  function  (Figure  14).  The  other  functions  called  in  'PROCEDURE 
Compute_response'  are  given  in  Figure  15.  The  sigmoidal  function  used  here  (a 
tanh  function)  for  the  function  f  in  equation  1  has  some  nice  properties,  but 
one  must  be  careful  not  to  generate  underflows  and  overflows  in  the  'EXP' 
function. 

We  used  the  variable  'synapses'  in  order  to  weight  the  afferent  and 


intracorneal  inpuca  Co  our  cells.  ICa  type  definition  ia 


aynapaea_type  “  RECORD 

afferent .intracortical  :  ARRAYlcellindex]  OF  maxvector. 

Thus,  ve  get  two  matrices,  'synapses. afferent'  and  'aynapaea. intracortical' .  We 
define  tbeae  matricea,  however,  aa  arrava  of  arrava  ('maxvector'  ia  defined  above 
a a  an  array  of  the  largeat  dimensionality  required),  in  order  to  eaaily  paaa  a 
a ingle  row  rather  than  all  the  rows  of  auch  a  matrix.  The  entire  matrix  ia  never 
uaed  all  at  once,  but  only  row  by  row.  Even  in  programming  languagea  which  do 
not  provide  auch  an  array  of  arraya  construction,  one  should  consider  stripping 
off  a  row  of  a  matrix,  putting  the  row  into  a  buffer,  and  working  with  the 
buffer,  rather  than  manipulating  the  entire  matrix.  On  the  other  hand,  it  an 
array  processor  ia  available,  the  software  should  take  advantage  of  the  enormoua 
efficiency  of  the  matrix  algebra  handling. 


STIAPTIC  MODIFICATION 


The  procedures  in  Figure  16  sketch  a  coding  for  the  heart  of  the  model,  the 
synaptic  modification  algorithms  (equations  2  and  3).  Here,  we  call  a  common 
routine  for  modifying  both  afferent  and  intracortical  connections,  but  completely 
different  modification  schemes  would  be  uaed  in  some  models,  while  many  models 
modify  only  one  of  these  pathways  (see  ref.  8).  As  mentioned  above,  the  time 
constants  in  the  modification  rules  (the  parameter  'b',  for  example)  determine 
tne  duration  of  a  single  iteration  of  the  time  loop  in  terms  of  the  duration  of 
a  stimulus  presentation  in  real  time.  To  make  a  connection  between  simulated 
time  and  real  time  one  would  have  to  speculate  far  beyond  current  knowledge.  We 


emphasize  that  the  output  of  the  simulation  vill  be  very  sensitive  to  the 
parameters  in  these  key  modification  routines.  The  values  for  these  parameters 
should  usually  be  set  interactively,  rather  than  during  compilation.  However,  a 
compl  te  investigation  of  the  behavior  of  these  models  requires  sensitivity 
analysis  (ref.  2),  and  systematic  variation  of  parameters  should  be  written  into 
a  program  at  some  point  for  this  purpose  (see  Discussion). 

AMALYSXS 

This  completes  the  simulation  of  the  model.  In  order  to  observe  and  study 
tne  simulation  at  intervals  within  the  time  loop,  we  occasionally  interrupt  the 
loop  in  order  to  test  the  network.  A  test  usually  consists  of  three  steps:  1) 
compute  the  cortical  cell  activities  using  a  standard  set  of  test  stimuli;  2) 
analyze  the  activities  to  make  explicit  the  relevant  measures  of  interest,  such 
as  selectivity  and  ocular  dominance;  and  3)  write  the  data  to  output  devices  and 
fite*.  This  can  be  a  tremendously  complex  task,  and  we  will  only  indicate  some 
of  the  basics.  Much  of  the  work  in  coding  program  outputs  involves 
device-dependent  routines,  especially  for  graphics.  The  procedure  in  Figure  17 
sketches  in  one  block  of  pseudocode  the  sorts  of  actions  needed.  Mote  that  the 
main  steps  replicate  the  simulation's  driver  (Figure  4),  with  the  differences 
being  that  stimuli  are  chosen  differently  and  the  synaptic  modifications  are 
bypassed.  The  following  example  (Figures  18-22)  makes  the  sketch  explicit. 

To  begin  the  analysis  we  need  to  create  a  standard  set  of  test  stimuli  and 
compuce  the  cell  responses  for  these  stimuli  (Step  1).  In  the  example  of  Figure 
18  the  testing  stimuli  are  a  subset  of  the  patterns  used  to  train  the  network. 

The  variable  'stimcount'  controls  the  manner  in  which  the  patterns  are  presented 
to  the  network.  For  'stimcount'  equal  to  0  the  system  tests  the  cortical  cells 


by  presenting  the  petterne  to  both  eyes  simultaneously.  For  'stimcount'  equal  to 


1  tne  system  presents  the  patterns  to  the  left  eye  and  nothing  to  the  right  eye, 
vhiie  for  'stimcount'  equal  to  2  the  system  presents  the  patterns  to  the  right 
eye  and  notbing  to  the  left  eye.  When  'stimcount'  equals  3,  the  system  leaves 
step  1  and  passes  the  raw  data  stored  in  variable  'analysis_data'  to  the  routine 
'Do_statistics' ,  which  comprises  step  2.  In  practice,  one  may  want  to  avoid  the 
storage  of  all  the  raw  data,  by  updating  the  statistics  as  each  test  stimulus  is 
presented. 

In  this  particular  routine  the  measures  of  interest  are  selectivity,  ocular 
dominance,  and  facilitation,  which  are  defined  within  the  routine  (Figure  19c). 
The  statistical  moments  that  the  routine  calculates  are  the  mean  and  variance  for 
each  measure  and  the  correlations  between  selectivity  and  ocular  dominance,  as 
well  as  facilitation  and  binocularity .  For  each  value  of  'eyetest' ,  or  more 
generally  of  'stimcount',  the  routine  must  determine  several  quantities:  the 
pattern  which  maximally  drives  each  cortical  cell  and  the  value  of  the  maximal 
response,  the  average  response  of  each  cortical  cell  over  all  stimuli  in  the 
test  set,  and  the  response  of  each  cortical  cell  when  the  pattern  that  drives  the 
cell  best  binocularly  is  presented  to  one  eye  at  a  time.  Using  these  quantities, 
the  routine  is  able  to  calculate  the  measures  of  interest  and  the  statistical 
moments.  This  completes  step  2. 

To  finish  the  analysis  the  data  must  be  sent  to  the  output  devices  and 
fixes.  This  section  of  the  routine  can  be  complicated,  particularly  if  a 
graphics  device  is  used  to  display  the  output  data.  In  this  example  the  routine 
simply  writes  the  results  to  the  terminal  in  tabular  form  (Figure  19d). 

The  teat  session  and  analysis  shown  here  is  entirely  deterministic. 
Stochastic  testing  is  also  of  great  interest.  Noise  can  be  injected  into  the 
system  in  a  number  of  places  (the  input,  the  synapses,  the  response)  and  the 


variability  of  the  reaponae  can  be  studied.  Unfortunately,  auch  tearing  requires 
even  more  run-time  and  analysis. 

Diaeoaaion 

Once  the  aimulation  ia  coded,  debugged,  and  compiled,  teat  ruaa  ahould  be 
performed  to  aaaeaa  the  moat  baaic  behaviora.  For  instance,  doea  the  ayatem 
remain  atable  alter  sufficient  experience  in  a  atable  environment?  Do  the 
responses  change  as  expected  under  monocular  rearing?  In  order  to  obtain 
satisfactory  results  from  these  baaic  simulations,  adjustments  will  need  to  be 
made  to  parameters.  The  numbers  of  atferenta  and  cortical  cells,  the  initial 
synaptic  weights,  the  amplitudes  of  the  inputs,  the  form  and  values  of  the 
function  f  in  equation  1,  the  parameters  in  the  modification  rules  (equations  2 
and  31,  and  the  sequence  of  stimuli  must  be  played  with  until  a  consistent  set  of 
parameters  becomes  evident.  There  is  no  good  substitute  for  experience  with 
observing  the  direct  effects  of  altering  these  various  inputs. 

However,  once  a  rough  feeling  is  obtained  for  when  the  simulation  runs 
reasonably,  a  great  deal  of  frustration  can  be  avoided  by  varying  the  large 
parameter  set  in  a  systematic  manner.  Since  a  typical  program  potentiallly 
contains  20  or  30  parameters,  preliminary  considerations  obtained  from  early  runs 
or  analysis  of  the  model  should  be  used  to  exclude  certain  of  these  parameters 
from  further  analysis  by  virtue  of  their  lack  of  significance  for  the  eventual 
results.  Those  parameters  which  may  affect  the  output  should  then  be  varied 
around  their  nominal  settings,  as  estimated  by  the  preliminary  runs. 

The  analysis  of  the  system  output  as  a  function  of  the  parameters  is  termed 
"sensitivity  analysis".  By  examining  the  results  of  a  large  number  of  runs  at 
different  parameter  values,  one  can  establish  the  sensitivity  of  the  output 


variable*  Co  each  parameter*  and  aC  Che  same  time  optimal  values  for  the 
parameters  can  be  located.  Cukier  et  al  (ref.  2)  developed  a  method  of 
sensitivity  analysis  which  applies  to  our  simulations.  A  large  number  of  runs 
are  required*  but  fever  than  with  a  brute  force  method  of  changing  one  parameter 
at  a  time  while  otners  are  fixed.  Their  method  varies  each  parameter 
sinusoidally  at  different,  independent  frequencies,  then  Fourier  analyzes  the 
system  output  back  into  each  of  the  independent  frequencies.  The  power  at  a 
given  frequency  and  it*  harmonics  indicates  the  sensitivity  of  the  output  to  the 
associated  parameter.  Unfortunately  *  the  range  over  which  a  parameter  is  varied 
influences  the  measure  of  sensitivity  obtained*  since  varying  a  parameter  over  a 
small  range  will  result  in  less  variation  of  the  output  than  with  a  larger 
range.  Inspection  of  the  data  obtained  from  the  many  runs  allows  one's 
judgments  of  the  nominal  ranges  to  be  improved*  along  with  improving  the  nominal 
values  upon  which  these  ranges  should  be  centered. 

As  an  example  of  the  usefulness  of  sensitivity  analysis*  consider  the 
dependence  of  orientation  selectivity  on  some  of  the  parameters  of  our 
simulation.  In  this  model*  we  begin  with  an  initial  state  of  low  selectivity* 
and  with  appropriate  experience  selectivity  increases  to  some  asymptotic  level. 
Our  measure  of  selectivity  takes  values  between  0  and  1*  with  0  corresponding  to 
a  flat  tuning  curve*  while  1  corresponds  to  a  tuning  curve  which  is  infinitely 
narrow  (i.e.  a  delta  function).  We  discussed  above  how  we  code  visual  stimuli 
which  represent  an  abstraction  of  oriented  bars*  and  pointed  out  that  the 
important  parameter  in  these  stimuli  is  the  overlap  between  different  stimulus 
vectors.  The  parameter  'vidth_of_input'  in  Figure  12  determines  this  overlap. 

If  the  overlap  is  large*  the  different  stimuli  will  look  alike  to  the  cortical 
cells*  which  should  therefore  show  less  selectivity  than  if  the  overlap  is 
smaller  and  the  stimuli  are  more  easily  distinguished.  By  performing  a 


sensitivity  analysis,  one  esn  investigate  the  extent  to  which  selectivity  depends 
on  this  input  tuning,  compared  to  the  dependence  on  other  parameters  such  as  the 
synaptic  modification  time  constants.  Furthermore,  one  series  of  runs  informs 
the  investigator  not  only  about  the  asymptotic  level  of  selectivity,  but  about 
tne  progression  of  the  sensitivity  with  experience.  Early  in  a  run,  selectivity 
might  be  found  to  depend  on  the  excitatory  modifications,  while  later  the 
inhibitory  modifications  become  more  important,  and  eventually  the  asymptotic 
level  of  selectivity  may  be  sensitive  to  the  width  of  the  input  tuning  as  well  as 
the  modification  speeds.  Of  great  importance  in  this  analysis  is  the  discovery 
of  parameters  to  which  the  output  is  insensitive.  If,  for  example,  the  system  is 
insensitive  to  the  parameters  involved  in  the  inhibitory  processes,  these 
processes  contribute  little  to  the  operation  of  the  model. 

Once  the  very  hard  job  of  finding  appropriate  parameters  is  completed,  the 
simulation  can  be  run  under  various  rearing  conditions  to  show  that  experimental 
results  can  be  replicated  (at  least  in  a  very  limited,  abstract  contextl)  and 
novel  rearing  conditions  can  be  used  to  "predict"  the  results  of 
yet- to-be-performed  experiments. 

The  main  value  of  running  simulations,  we  feel,  is  not  to  pretend  that  one 
is  imitating  the  nervous  system  in  any  detail,  but  instead  to  gain  an 
understanding  of  some  of  the  concepts  which  may  someday  help  to  model  the 
physiology  in  detail.  For  instance,  the  use  of  various  forms  of  Bebbian  synapses 
enables  very  powerful  computations  to  be  performed  by  parallel  networks,  with 
little  need  for  prior  organization.  Simulations  aid  in  appreciating  both  the 
power  and  the  limitations  of  the  concept  of  self-organization  by  synaptic 
modification.  For  example,  one  learns  quickly  to  pay  attention  to  input  coding, 
which  is  practically  ignored  in  most  models  prior  to  their  realization  in 
simulations.  Whereas  a  model  of  central  processes  might  assume  that  the 
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peripheral  nervoue  system  copiee  external  stimuli  faithfully,  a  simulation  night 
aaauae  that  the  central  nervoue  ay a ten  receivea  inputa  vhich  are  all  but 
completely  processed. 

Simulations  of  neural  ayatema  aerve  to  develop  applicationa  in  artificial 
intelligence.  One  auch  application  night  be  in  the  programming  of  parallel 
computera.  A  neural  network  with  modifiable  eynapaea  provide*  one  of  aeveral 
potential  architecturea  for  parallel  hardware,  and  knowledge  of  the  behavior  of 
theae  ayatema  will  allow  future  nachinea  to  be  uaed  to  not  only  aimulate  neural 
modela  at  high  apeeda,  but  alao  in  applicationa  where  continuoua  adaptive 
behavior  or  aaaociative  recall  of  large  databaaea  ia  required.  The  code 
deacribed  in  thia  article  waa  constructed  to  aimulate  a  self '“organizing  system, 
where  no  external  information  about  the  system  output  is  available.  However, 
this  program  can  be  adapted  to  run  a  supervised  learning  system,  where  the 
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desired  output  is  fed  to  the  synaptic  modification  algorithm  as  a  goal  (see  ref. 
71.  The  error  between  the  actual  output  (&)  and  the  desired  output  can  then  be 
uaed  to  modify  synaptic  weights.  Such  an  error-correcting  method  allows  the 
system  to  be  taught  to  respond  to  given  inputa  with  outputs  which  are  determined 
in  advance.  The  system  can  be  taught  to  discriminate  between  slightly  different 
inputa,  or  to  categorize  all  inputa  which  are  similar  to  some  prototype  as  a 
given  class.  Although  the  above  discussion  concerns  sensory  processing 
primarily,  one  could  substitute  a  motor  system's  anatomy  in  order  to  develop 
effector  instruments. 
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In  our  previous  article  (  )  we  discussed  models  of  the  development  of  visual 


cortical  specificity.  The  behavior  of  such  a  model  can  be  simulated  on  a  serial 


computer  by  stepping  through  a  sequence  of  stimulus  presentations  which  mimic 
visual  experience.  The  simulated  experience  controls  the  development  of  cortical 
responses  through  an  algorithm  which  changes  the  simulated  synaptic  weights. 

Such  simulations  provide  a  means  to  study  models,  and  should  be  regarded  more  on 
an  abstract  level  than  as  simulations  of  real  neural  processes. 
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Figure  Caption* 


Figure  1 : 

One  way  Co  generate  inpuc  aCiauli  is  co  project  bars  onto  a  sheet  of 
receptor  elements.  The  activity  of  each  receptor  is  proportional  to  the  degree 
to  which  that  receptor  is  covered  by  the  bar. 

Figure  2: 

Examples  of  abstract  stimuli.  The  curves  are  gaussians  on  a  circle  (see 
'FUNCTION  contour'  in  Figure  12).  Two  stimuli  are  shown,  centered  on  0  (solid) 
and  45  (dashed).  On  Che  left,  the  curves  are  displayed  in  polar  form,  with 
amplitude  given  by  the  radial  distance  from  the  circle.  On  the  right,  the  circle 
is  unwrapped  to  a  cartesian  plot  of  amplitude  versus  orientation  (or  input 
fiber).  The  overlap  between  two  stimuli  depends  on  the  distance  between  their 
centers  and  on  the  width  of  the  curves. 

Figure  3: 

A  flow  chart  for  a  simulation.  The  program  consists  of  a  loop  which 
simulates  tne  passage  of  time.  Following  initialization  of  variables,  for  each 
iteration  of  tbe  time  loop  an  appropriate  input  is  chosen.  This  input  induces  a 
response,  which  often  must  be  computed  iteratively.  Display  of  this  input/output 
relationship  may  be  useful,  and  we  indicate  this  by  the  Ngraphicsn  box.  Ac 
certain  points  in  the  time  loop  the  simulation  is  interrupted  for  analysis  of  tbe 
system.  The  analysis  routine  can  use  the  same  stimulus-generation  and 
response-computation  procedures,  by  setting  a  standard  sequence  of  test  stimuli. 
During  analysis  tbe  system  is  left  unaltered,  but  during  the  mam  simulation 
sequence,  synaptic  strengths  are  modified  based  on  the  stimulus  and  response. 


Data  la  written  to  output  devices  when  desired,  generally  before  each  time  step. 

Figure  4: 

The  driver  for  the  simulation.  When  the  driver  calls  the  routine 
'Choose_stimulus' ,  the  variables  'time',  random_seed' ,  and  'stimulua'  paas 
explicitly.  Notice  that  'PROCEDURE  Choose.stimulus'  changes  the  value  of 
'stimulus'  • 

Figure  5: 

Routine  for  determining  rearing  conditions.  The  'CASE  ( random_seed  MOD  10) 
OF'  statement  calls  the  routine  that  is  numbered  by  'rando^_seed  MOD  10'.  For 
example,  it  random_aeed  equals  128,  then  (random_seed  MOD  10)  equals  8  and  the 
CASK  OF  statement  calls  'PROCEDURE  Disparate'.  The  variables  'time'  and 
'randoia_seed'  must  be  specified  before  entering  'PROCEDURE  Choose_stimulus' . 
'PROCEDURE  Choose_stimulus' ,  however,  assigns  a  value  to  the  variable  'stimulua' 
and  passes  this  new  value  back  to  the  main  driver  of  the  program. 

Figure  6t 

Record  variable  for  storing  visual  stimulus  conditions.  The  variable 
'stimulus'  contains  a  'tag'  component,  which  takes  on  values  of  the  type 
'rearing',  and  for  each  value  of  the  'tag'  component  several  other  variables. 

For  example,  'tag  "  ad' ,  'open  “  right',  and  'pattern  “  5'  specifies  the  'tag' 
component  of  'stimulus',  i.e.  we  are  looking  at  the  *md'  component  of 
'stimulus',  and  the  values  of  the  variables  for  'tag  “  ad' ,  i.e.  'stimulus,  open  “ 
right'  and  'stimulus. pattern  “  S'. 


Figure  7 : 


Routine*  for  defining  rearing  condition*. 

Figure  8: 

Routine*  for  defining  rearing  condition*.  The  routine  'Procedure  Disparate' 
uaea  a  crude  method  to  generate  disparities  between  the  left  and  right  eye 
stimuli.  The  function  'random'  return*  a  number  between  0  and  1,  which  i*  then 
uaed  to  generate  'dispshift'  which  liea  between  -2  and  *2»  which  finally  leada  to 
tne  difference  between  'lef tangle'  and  'rightangle'  (the  disparity)  lying  between 
-4  and  +4.  However,  the  distribution  of  'dispshift'  is  not  uniform,  so  that 
disparities  tend  to  be  between  -2  and  +2. 

Figure  9: 

Routines  for  defining  rearing  conditions. 

Figure  lU: 

Translation  of  rearing  conditions  into  stimulus  vector.  The  main  driver  of 
the  program  paaaes  'stimulus'  and  'af ferent_activity'  explicitly  to  'PROCEDURE 
Code_atferent_input' .  The  value  of  'stimulus'  determines  the  pattern  types  that 
are  asaigned  to  the  vector  'afferent_activity'.  In  this  routine  we  have  two 
loopa.  The  firat  (aecond)  pass  through  the  outer  loop  assigns  activities  to  the 
components  of  'af ferent_activity'  corresponding  to  the  left  (right)  eye.  The 
inner  loop  calculates  the  activity  assigned  to  each  component  by  calling 
'FUNCTION  atf erent_component'  which  uses  the  tag  value  of  'stimulus'  to  set  the 
activity.  Notice  that  'FUNCTION  af ferent_component*  returns  a  numerical  quantity 
to  'PROCEDURE  Code_af f erent_input' .  In  general,  PASCAL  procedures  change 
variable  values,  whereas  PASCAL  functions  perform  some  calculation  and  return  the 
result  of  the  calculation  as  the  value  of  the  function. 


Figure  li: 


Function  which  rune  through  conditions  for  each  type  of  stimulus. 

Figure  12: 

Functions  to  set  activity  in  afferent  fibers. 

Figure  13: 

Iterative  computation  of  cortical  activity.  Variable  'used'  is  a  vector  of 
dimension  N,  the  number  of  simulated  cortical  cells.  Each  component  of  'used' 
carries  a  value  of  TRUE  or  FALSE,  where  TRUE  (FALSE)  indicates  that  the  procedure 
has  (has  not)  calculated  the  afferent  input  to  the  corresponding  cell.  The 
statement  'uaed  :■  all_falae'  sets  all  of  the  componenta  to  FALSE.  When  the 
REPEAT  loop  has  updated  each  cell  at  least  once,  all  the  components  of  'used'  are 
TRUE  and  the  implicit  updating  ends. 

Figure  14: 

Inner  product  routines. 

Figure  13: 

Functions  used  in  response  calculations. 

Figure  16: 

Synaptic  modification  routines.  Notice  that  modifying  the  synapses  takes 
very  little  code.  Each  pass  through  the  loop  of  'PROCEDURE  Modify.synapses' 
modifies  the  synapses  of  one  cell  by  calling  'PROCEDURE  Modify'  which  changes  the 
synaptic  weights  of  the  cell  one  by  one. 


Figure  17: 


Analysis  driver. 

Figure  18: 

Analysis  routine. 


Figure  19: 

Subroutine  for  analysis,  a) 
statistics:  d)  writing  output. 


initializations;  b)  preliminary  calculations;  c) 


Graphics 


Modify 

synapses 


Write 

Results 


FOR  time  :*  0  TO  end_of_time  DO 
BEGIN 

Choose.stimulus  ( time, rand om_seed , stimulus) ; 
Code_afferent_activity  ( stimulus ,afferent_activity) ; 
Compute  .response  (af ferent_activity , synapses, response) 
Modify.synapses  (afferent.activity .response .synapses) ; 
IF  time  IN  analysis.times  THEN  Analysis  (synapses); 
Log_results_to_outpu t.devices 
END; 


PROCEDURE  Choose_stimulus  ( time: times;  random^ seed: INTEGER; 

VAR  stimulus : stimulus_type) ; 

BEGIN 

IF  time  <  1000  TBEN  Dark  ( random_seed .stimulus) 

ELSE 

CASE  (rsndon^_seed  MOD  10)  OF 

0(1  :  Dark  ( rsndom_seed, stimulus) ; 

2  :  Closeleft  (random_seed, stimulus) ; 

3  :  Closeright  ( random_seed, stimulus) ; 

4  :  Strabismic  ( random_seed, stimulus) ; 

5.6, 7,8, 9  :  Disparate  (rsndom_seed, stimulus) 


•  tunul.ua_.type  “ 


RECORD 

CASE  tag  :  rearing  OF 

correlated  t  (orientation  :  angle); 
ad  :  (open  :  eyes;  pattern  :  angle); 
ra  :  (firatopen  :  eyes;  reversal  :  times; 

bar  :  angle); 


dr  s  (); 
disparity  : 
strabismus 
adaptation 

periodic  : 


(rightangle.lef tangle  :  angle); 

:  (r tangle ,1 tangle  :  angle); 

:  (adapted_eye  :  eyes; 

adapting  pattern  :  angle) ; 
(frequency  :  real;  phase  :  angle) 


PROCEDURE  Normalrearing  ( randon_«eed  j  INTEGER; 

VAR  atimulua  :  atimulua_type) ; 

BEGIN 

WITH  atimulua  DO 
BEGIN 

tag  :■  correlated; 

orientation  :■  1  ♦  ( random_«eed  MOD  nuaanglea) 
END 

END;  (Normalrearing) 


PROCEDURE  Cloaeleft  (random_aeed  :  INTEGER; 

VAR  atimulua  :  atimulua_type) ; 

BEGIN 

WITH  atimulua  DO 
BEGIN 

tag  :■  ad; 
open  :■  right; 

pattern  ;•  1  +  (random^aeed  MOD  numanglea) 
END 

END;  (Cloaeleft) 


PROCEDURE  Cloaeright  ( random_aeed  :  INTEGER; 

VAR  atimulua  :  stimulua_type) ; 

BEGIN 

Cloaeleft  (random_aeed, atimulua) ; 
atimulua. open  :■  left 
END;  (Cloaeright) 


PROCEDURE  Reverae_auture  (random_aeed  :  INTEGER; 

VAR  atimulua  :  atimulua.type) ; 

BEGIN 

WITH  atimulua  DO 
BEGIN 

tag  :«  ra; 
firatopen  :■  right; 

reveraal  !"  2000;  (or  whatever  time  ia  deaired) 
bar  :■  1  ♦  ( random_aeed  MOD  numanglea) 

END 

END;  (Reverae.auture) 


PROCEDURE  Dark  ( r*ndo*_»eed  t  INTEGER; 

VAR  etiaulue  :  atiaulua_type) ; 

BEGIN 

atiaulue.tag  :■  dr 
END;  {Darkrearing} 


PROCEDURE  Diaparate  (randoa_»eed  :  INTEGER; 

VAR  aciaulua  :  atiaulua_type) ; 

CONST 

diapwid  ■  3*2; 
diapcrana  ■  1.6; 

VAR 

diapahift  :  INTEGER; 

BEGIN 

WITH  aciaulua  DO 
BEGIN 

tag  :•  diaparity; 

rightangle  :•  1  ♦  ( random_aeed  MOD  nuaanglea); 

IP  rightangle  >  nuaanglea  DIV  2 

{  Only  half  the  patterna  (the  "vertical"  onea)} 

{  induce  diaparitiea.  ) 


THEN 

BEGIN 

diapahift  : 
leftangle  : 

diapahift  : 
rightangle 

END 

ELSE 

leftangle  : 

END 

END;  (Diaparate) 


ROUND(diapwid  *  randoa(randoa^aeed)  -  diaptrana); 
1  +  (rightangle  +  nuaanglea  —  1  +  diapahift) 

MOD  nuaanglea; 

ROUND(diapvid  *  randoa(randoa_®eed)  -  diaptrana); 
i  i  +  (rightangle  ♦  nuaanglea  —  1  ♦  diapahift) 

MOD  nuaanglea 


rightangle 
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PROCEDURE  Strabismic  ( random_seed  s  INTEGER; 

VAR  stimulus  :  stiaulus_type) ; 

BEGIN 

WITH  stimulus  DO 
BEGIN 

tag  :■  strabismus; 

rtangle  :■  1  ♦  (randota_seed  MOD  numangles); 

1 tangle  :■  1  ♦  TRUNC( numangles  *  random( random_seed) ) 

END 

END;  {Strabismic} 

PROCEDURE  Adapting  (random_seed  :  INTEGER; 

VAR  stimulus  :  stimulus.type) ; 

BEGIN 

WITH  stimulus  DO 
BEGIN 

tag  :■  adaptation; 
adapted_eye  !“  right; 
adapting  pattern  :■  17 

END  {This  is  an  example.  Any  stimulus  can  be  fixed  here.} 
END;  {Adapting} 


PROCEDURE  Sinusoid  (random_seed  :  INTEGER; 

VAR  stimulus  :  stimulus_type) ; 

BEGIN 

WITH  stimulus  DO 
BEGIN 

tag  :■  periodic; 

phase  :•  1  ♦  (random_seed  MOD  numangles); 
frequency  :■  two_pi_over_n  *  (random_aeed  MOD  10) 

END 

END;  {Sinusoid}  {  two_pi_over_n  ■  (2  *  PI)  /  numangles  } 
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£  PROCEDURE  Code_a£f erent_input  (stimulus  :  stimulus_type; 

S  VAR  af ferent_sctivity  :  maxvector); 


VAR 

fiber, 

f iber_number  :  maxindex; 
left_or_right  :  eyes; 

BEGIN 


{  Numaf ferents  equals  the  total  number 
{  of  afferent  fibers  to  each  cell. 

{  maxindex  “  1 ..numaf ferents 
{  or  l..numcells 

{  depending  on  which  is  larger. 

{  maxvector  “  ARRAY  [maxindex]  OF  REAL 


f iber_number  :■  0; 

FOR  lef t_or_right  :■  left  TO  right  DO 
FOR  fiber  1  TO  (numaf ferents  /  2)  DO 

{  We  take  > 

{  numangles  "  numaf ferents  /  2.  > 

{  That  is,  each  pattern  (or  angle)  corresponds  ) 
{  to  an  afferent  fiber  from  each  eye.  } 

BEGIN 


fiber_number  :■  fiber_number  +1; 
afferent_activity[fiber_number]  :■ 

af  f  erent_ac  t  ivity_component(  lef  t_or_right,  fiber,  stimulus) 

END 

END;  (Code_afferent_input) 


> 

) 

> 

> 

} 

} 


FUNCTION  aff erent_component  ( lef t_or_right  :  eye*; 

fiber  :  max index;  stimulus  :  stimulus_type)  :  REAL ; 

VAR 

x:  REAL; 

timerev, lropen  :  BOOLEAN; 

BEGIN 

WITH  stimulus  DO 
CASE  tsg  OF 

correlated  : 

x  :■  contour( fiber , orientation) ; 
md  : 

IF  lef t_or_right  ■  open 

THEN  x  :■  contour( fiber, pattern) 

ELSE  x  :**  noise(noise_amplitude) ; 

rs  : 

BEGIN 

timerev  : “  time  <  reversal; 
lropen  :■  lef t_or_right  “  firstopen; 

IF  (NOT  timerev  OR  lropen)  AND  (timerev  OR  NOT  lropen) 
(timerev  IFF  lropen} 

THEN  x  :■  contour( fiber, bar) 

ELSE  x  !■  noise(noise_amplitude) 

END; 
dr  : 

x  :■  noi#e(noise_amplitude) ; 
disparity  : 

IF  left_or_right  •  left 

THEN  x  :■  contour( fiber, lef tangle) 

ELSE  x  :■  contour( fiber, rightangle) ; 

strabismus  : 

IP  lef t_or_right  «  left 

THEN  x  ;■  contour( fiber, ltangle) 

ELSE  x  :■  contour( fiber, rtangle) ; 

adapting  : 

IF  (adapted.eye  ■  both)  OR  ( lef t_or_right  "  adapted_eye) 
THEN  x  :■  contour( fiber .adapt ing_pat tern) 

ELSE  x  :■  noise(noise_amplitude) ; 

periodic : 

x  :■  (l«C0S(frequency*(fiber-phase)))/2 

END; 

afferent^component  :■  x 
END;  {afferent_component} 


FUNCTION  contour  (fiber  :  maxindex;  center.angle  :  angle)  :  REAL; 


CONST  {  Plotting  fiber  activity  veraua  fiber  ) 

v id th_of .input  ■  1.0;  {  number  givea  a  Cauaaian  curve  with  the  } 

{  peak  value  occurring  at  'center.angle'.) 
{  (aee  Figure  2)  } 

BEGIN 


contour  :■  EXP( -v id th.of .input  * 

diatance_aquared( fiber ,center_angle)) 

END;  (contour) 


FUNCTION  diatance.squared  (fiber  :  max index;  center.angle  :  angle)  :  REAL 

{  Thia  function  calcuiatea  the  diatance  between  ) 

{  points  lying  on  a  unit  circle.  ) 

BEGIN 

diatance.squared  :■  1.0  -  C0S(  two _pi_over_n  *  ( f iber-center.angle)) 
END;  (diatance.squared) 

{  two_p i_ov e r.n  ■  2  *  PI  /  numangles  ) 


FUNCTION  noise  (amplitude  :  REAL)  :  REAL; 

BEGIN 

noise  :■  amplitude  *  random  (randoi*.seed) 
END;  (noise) 


FUNCTION  random  (VAR  random.seed  :  INTEGER)  :  REAL; 

(random  is  uniformly  distributed  between  0  and  1} 


CONST 

alpha  “  779;  {  See  refs.  4,5  on  the  generation  ) 

lambda  “  361;  (  of  random  variates.  } 

pea  “  16384; 

BEGIN 

random  :■  rand om_seed/ pea; 

random_seed  :■  (alpha  *  random.aeed  ♦  lambda)  MOD  pea 
END;  (random) 


A* 


PROCEDURE  Compu te_reaponae  (af ferent_activity  :  maxvector;  aynapaea  :  aynapaee_type; 

VAR  reaponae  :  maxvector); 


uaed  :  ARRAY  Icellindexl  OF  BOOLEAN;  {  cellindex  -  1.. numcella  > 

BEGIN 

uaed  :■  all_£alae; 

REPEAT 

ceil  :■  random_integer_between(l, numcella); 

IF  af ferent_input_haant_already_been_eomputed(cell) 

THEN  Compute_afferent_input; 
uaed (cell J  TRUE; 

Compu  te_intracor t ic  al_inpu t ; 
reaponaelcelll  :* 

aigmoid(af ferent_input[cell]  -  intracortical_inputlcelll) ; 

UNTIL  uaed  ■  all_true 
END;  {Compu Ce_reaponae} 


PROCEDURE  Compute_.afferent_i.nput; 


BEGIN 

afferent_input[cell]  :■ 

inner_product(afferent_activity, synapses, afferent [cell] ,nuoaf ferents) 

END; 


PROCEDURE  Compute_intracortical_input ; 

BEGIN 

intracorticaI_input[cell]  :■ 

inner_produc t( response, synapses. intracortical [cell] .numcells) 

END; 


FUNCTION  inner_produc t  (x,y  ;  maxvector;  lastindex  :  max index)  s  REAL; 
VAR 

ip  :  REAL; 
index  :  max index; 

BEGIN 

ip  0.0; 

FOR  index  :■  1  TO  lastindex  DO 
ip  :■  ip  +  x[ index]  *  y[ index]; 
inner_produc t  s"  ip 
END; 


FUNCTION  random_integer_.be tween  («,b  :  INTEGER)  :  INTEGER; 

VAR 

interval  :  INTEGER; 

BEGIN 

interval  :■  b-a+1; 
random_integer_between  :■ 

a  +  ROUND( interval  *  random( random_seed) )  HOD  interval 

END; 

FUNCTION  aigmoid  (x  :  REAL)  :  REAL; 

CONST 

threshold  *3.0; 
steepness  “  2.2; 
too_amall  *  -20.0; 
too_big  -  10.0; 

BEGIN 

IF  steepness*( threshold  -  x)  <  too.small 
THEN  aigmoid  1.0 

ELSE  IF  steepnesa*( threshold  -  x)  >  too_big 
THEN  sigmoid  :■  0.0 

ELSE  sigmoid  : “  1 .0/C  1 .0  ♦  EXP( steepness  *  (threshold 


PROCEDURE  Modify .synapses  (afferent.activity .response  :  maxvector; 

VAR  synopses  :  synapses. type) ; 


EEC  IN 

FOR  cell  :■  l  TO  numcells  DO 
BEGIN 

Modify  ( numaf ferents, afferent.activity, responsetcelll ,sf ferent .parameters, 
synapses. sfferentlcell ] ) ; 

Modify  (numcells .response .response [cell] , intracortical parameters, 
synapses. intracortical[cell]) 

END 

END;  (Modify.synapses) 


PROCEDURE  Modify  (lastindex  :  maxindex;  presynaptic  :  maxvector; 
postsynaptic."parameters"  :  REAL; 

VAR  junctions  :  maxvector); 

BEGIN 

FOR  index  :■  1  TO  lastindex  DO 

junctions! index]  :■  "the  appropriate  function  of"  (6) 

( presynaptic .postsynap tic .parameters) 


(e.g.  junctions l index]  :■  (1.0  -  h)  *  junctions] index] 

♦  h  *  presynaptic] index]  *  postsynaptic  ) 

END;  (Modify) 


PROCEDURE  Analysis  (synapses  :  synapaes_type) ; 
BEGIN 

inicialize  output  variables 
REPEAT 

choose  stimulus 
make  input  vector 
compute  response 

log  response  into  output  variables 
UNTIL  ail  desired  patterns  have  been  teated 
compute  averages  and  other  output  functions 
write  results  and  draw  graphics 
END; 


TYPE 

cell_analy sis  :  AEfiAYleyes.ee 11 index, angle]  OF  REAL; 

VAR 

stimeount  ;  INTEGER; 
yes  :  BOOLEAN; 
eyetest  :  eyes; 
orientest  :  angle; 
analysis_data  :  cell_analysia; 

BEGIN 

askf oranaly sis( time, yes) ; 

IF  yes  THEN 
BEGIN 

sCimcounC  :*  0; 

REPEAT 
BEGIN 

FOR  orientest  :•  1  TO  nuaangles  DO 
BEGIN 

WITH  stimulus  DO 
BEGIN 

IF  stimeount  “  0 

THEN  tag  :■  correlated 
ELSE 

IF  stimeount  <  3 
THEN  tag  :■  md; 

CASE  tag  OF 
correlated  : 

BEGIN 

orientation  :*  orientest; 
eyetest  :•  both 
END; 
md  : 

BEGIN 

pattern  :*  orientest; 

IF  stimeount  ■  1 
THEN 

open  : “  left 
ELSE 

open  :■  right; 
eyetest  :*  open; 
noiseamplitude  :*  0.0 
END 

END  {CASE  tag} 

END;  {WITH  stimulus} 

Code_afferent_activity  ( stimulus ,afferent_activity) ; 
Compute_response  (aff erent_activity .synapses, response) ; 

FOR  cell  :■  1  TO  numcells  DO 

ana ly sis_data( eye  test .cell .orientest ]  :■  response [cell 1 ; 
END;  {FOR  orientest) 
stimeount  :•  stimeount  ♦  1 
END  {REPEAT} 

UNTIL  (stimeount  ■  3); 

Do_statistics(analy8is_data) ; 

END  {yea} 

END;  {Analysis} 


{  'PROCEDURE  Analysis'  is  used  only  when 
{  the  variable  'time'  equals  a  time 
{  specified  in  'PROCEDURE  aakforanalysia 
{  which  we  omit  here. 


PROCEDURE  Do_statistics(analysis_data  :  cell_analysia) ; 


CONST 

fldvd  -  7; 
prec  “  2; 

TYPE 

analyint  “  ARRAY[eyes,cellindex]  OF  INTEGER; 
analydaCa  “  ARRAY (eyea, cell index]  OF  REAL; 
overcells  ■  ARRAY [eyes]  OF  REAL; 

VAR 

eyeteaC  :  eyes; 
orientest  :  angle; 
best_acimulus  :  analyint; 
max,mean,sel,resp_at_binoc_max  :  analydata; 
ocdom, facilitation, responsiveness  :  maxvector; 
mean_ael , weighted_mean_sel , variance_sel  :  overcells; 
mean_ocd om, weigh ted_mean_ocdom,variance_od , 
mean_f acil , weighted_mean_f acil ,variance_f ac , 
mean_responsiveneas , 

correlation_sel_od, correlation^f ac_b inocular ity 


BEGIN 

{  initializations:  > 

FOR  eyetest  left  TO  both  DO 
BEGIN 

FOR  cell  :■  1  TO  numcells  DO 
BEGIN 

max[eyetest,cell]  :■  0*0; 
mean(eyeteat,cell j  :■  0.0; 
responsiveness [cell]  :■  0.0 
END;  (FOR  cell} 
mean_sel(eyetest]  :■  0.0; 
weighted_meao_sel[eyetest ]  : “  0.0; 
variance_sel[eyetest]  :*  0.0; 

END;  (FOR  eyetest} 
mean_ocdom  :■  0.0; 
weighted_mean_ocdom  :■  0.0; 
variance_od  ;■  0.0; 
mean_facil  :■  0.0; 
weighted,j&ean_facil  :■  0.0; 
variance_fac  :■  0.0; 
mean_responsiveness  :■  0.0; 
correlation_8el_od  :■  0.0; 
correlation_fac_bihocularity  :■  0.0; 


{  End  of  initializations  } 


FOR  cell  1  TO  nuacelle  DO 
BEGIN 

FOR  orienteet  :■  1  TO  nuacelle  DO 
BEGIN 

IF  aax[botb,cell]  <  analyeie_data[both, cell, orienteet] 

THEN 

BEGIN 

aax[both,cell]  :■  analyeie_data[both, cell, orienteet] ; 
beet_etiaulue(both,cell]  :■  orienteet 
END;  {IF  aex> 

END;  (orienteet) 

FOR  eyeteet  :■  left  TO  right  DO 
BEGIN 

FOR  orienteet  :■  1  TO  nuacelle  DO 
BEGIN 

IF  aaxf eyeteet, cell]  <  analyeie_data[eyetest,cell, orienteet]  THEN 
BEGIN 

aax[ eyeteet .cell]  :»  analyeie_data[eyeteet,cell, orienteet] ; 
beet_etiaulue[eyeteattcell]  :■  orienteet 
END;  {IF  aez) 

IF  orienteet  “  beet_etiaulue[both,cell] 

THEN  reep__at_binoc_pax[eyeteet,cell]  : “ 

enely eie_data[ eyeteet .cell. or ienteet ] ; 
mean[eyeteet,cell]  :•  mean[eyeteet,cell] 

♦  analyeie_data[ eyeteet ,cell, orienteet] 

END  {FOR  orienteet) 

END  {FOR  eyeteet) 


END;  {FOR  cell) 


BEGIN 

FOR  eyeteat  :■  left  TO  both  DO 
BEGIN 

meanleyeteat.cell]  :■  meanleyetest.celll/numanglea; 

{  Definition:  } 

aelleyeteat.cell]  :■  1.0  -  meanleyeteat.celll/maxleyetest.cell] ; 
mean_sel[eyeteat )  :■  mean_sel[eyetest ]  ♦  aelleyeteat.cell] ; 

IF  reaponaiveneaa[cell]  <  maxleyeteat.cell] 

THEN  responsiveness [cell!  :*  maxleyeteat.cell] 

END;  {FOR  eyeteat) 

FOR  eyeteat  left  TO  both  DO 

weighted_mean_ael(eyeteat]  :■  weighted_mean_ael( eyeteat]  ♦ 
selleyeteat.cell]  *  reaponaiveneaalcell] ; 

{  Definition:  } 

ocdomlcell]  :■  max[right,cell]/(max( left, cell]  ♦  maxlright .cell] ) ; 
mean_ocdom  : ■  aean_ocdom  ♦  ocdomlcell] ; 
weighted_mean_ocdom  weigh  ted_mean__ocdom  ♦ 

ocdomlcell]  *  reaponaiveneaalcell]; 
variance_od  :■  variance_od  ♦  SQR( ocdomlcell]) ; 

{  Definition:  } 

facilitationlcell]  :■  resp_at_bini_maxlboth,cell]  / 

(resp_at_binv_max[ left, cell]+resp_at_bin_max[right, cell]) ; 
mean_facil  :■  mean_facil  ♦  facilitationlcell]; 
weighted_mean_f acil  : “  weigh ted_mean_facil  + 

facilitationlcell]  *  reaponaiveneaalcell]; 
variance__fac  :■  variance_fac  ♦  SQR( facilitationlcell]) ; 
mean_re8ponaivene8a  :■  mean_reaponaivenesa  ♦  reaponaiveneaalcell] 

END;  {FOR  cell) 

FOR  eyeteat  :•  left  TO  both  DO 
BEGIN 

mean_ael[eyeteat]  :-  mean_aelleyeteat]/numcella; 
veighted_mean_aelleyeteat] 

weighted_mean_sel[eyetest ]/mean_reaponsiveneBa 
END;  {FOR  eyeteat) 

mean_ocdom  :  ■  mean_ocdom/ numcella; 

weighted_meao_ocdom  :■  veighted_mean_ocdom/mean_responsivene8s; 
mean_facil  :■  mean_facil/numcella; 

weighted_mean_facil  :•  weighted_mean_facil/mean_responsiveness; 
mean_responsiveneas  :■  mean_responsiveneas/nuacella; 

FOR  eyeteat  : “  left  TO  both  DO 
variance_8el(eyeteat]  :• 

( variance_ael ( eyeteat ]  -  numcella  *  SQR(mean_aelleyeteat]))/(numcella-l) ; 
variance.od  :»  (variance.od  -  numcella  *  SQR(mean_ocdom))/(numcella-l) ; 
variance.fac  :-  (variance.fac  -  numcella  *  SQR(mean_facil) )/(numcells-l ) ; 

FOR  cell  :■  1  TO  numcella  DO 
BEGIN 

correlation_ael_od  : “  correlation_ael_od  ♦ 

(aellright,cell]-mean_aellright])  *  (ocdomlcell] -mean_ocdom) ; 
correlation_f ac_binocularity  :■  correlation_fac_binocularity 

♦  (mean_facil-facilitationlcell ] )  *  aba(ocdom[cell]  -  0,5) 

END;  {cell) 

correlation_ael_od  :-  correlation_ael_od/SQRT( variance_ael  *  variance_od) ; 
corre 1 at ion_fac_b inocular icy  :•  correiation_fac_binocularity  / 

SQRT( var iance.f ac  *  variance_od) ; 


vriteln; 

write ln(  'aelectivity':23 ,'od':9,'fac':8,'reap':7 , 

'pref  pattern' :23); 

writeC'cell  left  right  binoc  ':29); 
vriteln('  left  right  binoc' :49); 

FOR  cell  :■  1  TO  numcells  00 
BEGIN 

write(cell:5,sel[ lef t.cell] :fldvd :prec) ; 

write( sellright, cell] : fldwd :prec,sel[both, cell] : fldwd :prec) ; 
wri te(ocdom( cell] :fldwd :prec( facilitation [cell] : fldwd :prec) ; 
write( responsiveness [cell] : fldwd spree) ; 

write(best_stimulus[lef t.cell] ,best_stimulus[right,cell]) ; 
writeln(best_stimulus (both, cell]) ; 

END;  (cell) 

writeln; 

write ( 'avrgs  ' : 5 ,mean_ael( lef t] : f ldwd :prec, mean_.se  l  [ right ] : f ldwd :prec) ; 
wri te(mean_sel[ both] : fldwd :prec,mean_ocdom: fldwd :prec) ; 
wri te( mean__f acil : f ldwd :prec ,mean_responsivene8s : f ldwd :prec) ; 
writeln; 

write( '  ':5 ,weighted_mean_sel[ lef t] : fldwd :prec) ; 

wri te(weighted_mean_sel[ right] : fldwd :prec, 

weighted joiean_8el[both] :  fldwd  :prec)  ; 
write(weighted_mean_ocdom: fldwd :prec , weighted _mean_facil : fldwd : prec) ; 
write('  weighted  by  responsiveness') ; 
writeln; 

write('  standard  deviation  of  ocular  dominance  is  ' , 

SQRT(variance_od) : 1 0 : 4) ; 

writeln; 

writeln; 

wri te( 'correlations:  between  selectivity  in  right  eye  and  ':52); 

write ln( 'shift  toward  right  eye  :',correlation_sel_od: fldwd :prec); 

write( '  ':15, 'between  facilitation  and  binocularity  :'); 

write ln(correlation_fac_binocularity: fldwd :prec) ; 

writeln; 

writeln 


END;  {Do_statistics) 


